REPRODUCING SUBGROUPS OF SP(2,R). 
PART II: ADMISSIBLE VECTORS. 

G. ALBERTI, F. DE MARI, E. DE VITO, AND L. MANTOVANI 

Abstract. In part I we introduced the class £2 of Lie subgroups of S'p(2,M) and obtained a 
classification up to conjugation (Theorem 1.1). Here, we determine for which of these groups the 
restriction of the metaplectic representation gives rise to a reproducing formula. In all the positive 
cases we characterize the admissible vectors with a generalized Calderon equation. They include 
products of ID-wavelets, directional wavelets, shearlets, and many new examples. 



1. Introduction 

In this paper we complete the program begun in Part I, where we introduced the class £2 of 
Lie subgroups of S'p(2,]R), consisting of semidirect products S x if, where the normal factor E is 
a vector space of 2 x 2 symmetric matrices and if is a connected Lie subgroup of GL(2,M). We 
then obtained a classification up to Sp{2, ]R)-conjugation, which is restated below in Theorem 13 in 
more explicit notation. Here we prove which of the groups in the class £2 are reproducing, namely 
for which of them there exists an admissible vector r] G L'^{E?) such that 



(1-1) 11/11= J\{f,U,v)\'dg 

G 

holds for every / G L^(]R^), where dg is a fixed left Haar measure on G, and U is the restriction 



to G of the metaplectic representation. This is the content of Proposition 14 and of Theorem 15 
Secondly, and most importantly, we want to describe all the admissible vectors in the reproducing 
cases. Both problems are solved by means of the general theorems proved in [T]. We also prove 
an auxiliary result concerning what we call orbit equivalence, a notion designed in order to find 
conditions under which two groups that are not conjugate within Sp{2,'R) (hence not equivalent 
according to the general representation-theoretic principles that are relevant in this context) do 
exhibit the same analytic features, in the sense that they have coinciding sets of admissible vectors. 
Orbit equivalence, which is an analytic condition, allows us to treat several families in a concise 
way. In some sense, it should be thought of as a version of a change of variables. It is worthwile 
observing that all but one families depending on a parameter (precisely: those listed as (2D.1), 
(2D.2), (2D.3), (3D.4) together with (3D.5), (3D. 6) and (3D.7)) are shown to consist of orbitally 
equivalent groups. The exception is notably the family (4D.4) which is the family of shearlet 
groups, indexed by the parameter a G (—1,0] (the case a = corresponding to the usual shearlets 
with parabolic scaling), for which we show that no orbit equivalence can possibly exist. 

In the end, the most interesting new phenomenon is perhaps the appearence of four non- 
conjugate classes of two-dimensional groups. Each of them is isomorphic to the affine group, 
but the metaplectic representation restricted to each of them is of course highly reducible and 
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clearly not equivalent to the standard wavelet representation, the most obvious difference being 
that the former analyses two-dimensional signals and the latter acts on L^(]R). 

A broader discussion concerning the role of the metaplectic representation and of the groups in 
82 in signal analysis is to be found in Part I. Other significant contributions in this circle of ideas 
are contained in the recent papers [2] and [3]. 

The paper is organised as follows. In the rather long Section |2l after introducing some notation, 
we review the results contained in [IJ in some detail. In Section |3] we discuss orbit maps and intro- 
duce the notion of orbit equivalence, which will be used later. Here the main result is Theorem 12 
Section |4] contains the classification result and the equations for the admissible vectors. 



2. Notation and known results 

In this section we recall the main results concerning the voice transform associated to the 
metaplectic representation restricted to a suitable class of Lie subgroups of S'p(2,]R). We review 
rather thoroughly the results contained in pLj because we need a much simpler formulation here, 
which would not be plain to infer from pQ at first reading. 

2.1. Notation. Given a locally compact second countable space X, Cc{X) is the space of contin- 
uous complex functions with compact support. A Radon measure on X is a positive measure 
defined on the Borel cr-algebra of X and finite on compact subsets, and L^(X, i/) is the Hilbert 
space of the complex functions on X which are square integrable with respect to v. The norm 
and the scalar product of L'^{X, v) are denoted by and (■, ■)^, respectively, and if no confusion 
arises the dependence on v is omitted. 



2.2. Reproducing groups of Sp{d,M.). Given an integer c? > 1, we denote by Sp{d,M.) the 
symplectic group acting on M?''' and by /i the metaplectic representation of Sp{d, M) (see for example 
We recall that is a projective, unitary, strongly continuous representation of Sp{d, M) acting 
on L^(]R'^, (ix), where dx is the Lebesgue measure of W^. Given a Lie subgroup G of Sp{d^M.), we 
denote by dg a left Haar measure of G, by Ac its modular function, and by U the restriction of ^ 
to G. 

Fix now a Lie subgroup G of Sp{d, M). Regarded the space L^(]R'^, dx) as the set of d-dimensional 
signals, we introduce the voice transform associated to the representation U of G. Recall that a 
voice transform is obtained by choosing an analysing function r] G L'^{W^^dx), and then defining 
for any signal / G Ll^iW^^dx) the continuous function 

y,/:G^C, V,f{g) = {f,Ugr^). 

For an arbitrary G it may well happen that Vr^f is not in L'^{G, dg). The following definition selects 
the family of subgroups for which the voice transform becomes an isometry. Notice that we do not 
require U to be irreducible. 



Definition 1. A Lie subgroup G of Sp{d,M.) is a reproducing group if there exists rj G L'^{M.'^,dx) 
such that formula (1.1) holds for every / G L'^{R'^,dx). The function rj is called an admissible 
vector for G. 
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Under these circumstances, V^/ is in L'^iG, dg) and the analysis operator / i— )■ V^/ is an isometry 
from L'^{W^,dx) into L'^{G,dg) intertwining U with the left regular representation of G. Further- 
more, the following reproducing formula holds true for all / G L'^(M.'^, dx) 

(2.1) f = j {f,U,r^) U.vdg, 

G 

where the integral must be interpreted in the weak sense. 

Remark 2.1. Take a reproducing group G and g G Sp{d,M.). It is well-known that the conjugate 
group gGg^^ is reproducing, too. Furthermore, if ?7 G L^{M.'^,dx) is an admissible vector for G, 
then figi] is an admissible vector for gGg~^. 

2.3. The class S. In this paper, we consider subgroups of S'p((i, M) whose elements are "triangu- 
lar" d X d-block matrices. For a discussion on this choice the reader is referred to Part I. 

Definition 2. A Lie subgroup G of ^^((i, M) belongs to the class S if it is of the form 

h 



G 



{ ah ^^^}' 



where if is a connected Lie subgroup of GL(d, M) and S is a subspace of the space Sym(d, M) 
oi d X d symmetric matrices. We further require that both S and H are not trivial. Whenever 
needed, we write Sd to specify the size. 

In order for G to be a group it is necessary and sufficient that = S for all /i G if, where 

(2.2) h^[a]='h~^ah~\ 

U G E S, both S and H are naturally identified as Lie subgroups of G. Clearly, Sii = G, 



E n ii = {e}, S is a normal subgroup of G and it is invariant under the action of H given by (2.2 ), 
so that G is the semi-direct product S x ii. 

For the remaining part of this section, we fix a group G = S x ii in the class S. A left Haar 
measure and the modular function of G are 

(2.3) dg = xih)-'dadh Aaia, h) = xihy^Anih), 

where da is a Haar measure of S, rf/i is a left Haar measure of H, Ah is the modular function of 
H, and x is the positive character of H given, for all h E H, hj 

(2.4) x{h) = \det a ^h^[a]\. 



We denote by S* the dual of S. The contragredient action of (2.2) is then given by 



(2.5) h[a*]{a) = a*{{h-y[a]) = a*Chah), a* E J:* , a E J:, h E H. 



Since the action (2.5) will play a more relevant role than the action (2.2), we have chosen the 



simpler notation for the former. 
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2.4. The representation U. The restriction U of the metaplectic representation to G G £^ is 
completely characterized by a "symbol" $, as we now explain. In its standard form, U acts on 
/ G L'^iR'^) by 

(2.6) U^.,h)fi^) = Pih)-"^ e"'<'^^'"V(/i~'a;), a.e. x G W\ 
where (3 is the positive character of H given by 

P{h) = \deth\. 

Now, given x G M"', the map cr i— ?■ — | {ax,x) is a linear functional on S and hence it defines a 
unique element G S* by the requirement that for all cr G S 

(2.7) $(a:)(a) = -^ (^a;,a;)j,,. 

The corresponding function $ : M'^ — )■ S* has a fundamental invariance property. Indeed, noting 
that the group H acts naturally on M*^ by means of 

h.x = hx X eW'-, h e H, 

it follows that for all x G M*^ and h G H, 

(2.8) (!>{h.x) = /i[$(x)]. 
This is seen by observing that for all a G S we have 

^h.x){a) = {'hahx,x)^, = ^x)iih-y[a]) = /i[$(x)](a). 
Therefore, for o" G S and h ^ H,hj 

Ui.,H)f{x) = /3(/i)-i e-2-^*(^')('^)/(/i-ix), 
which exhibits f/ as a representation of the kind considered in [1], with the quadratic symbol $. 

2.5. More notation. In the examples, we often parametrise the n-dimensional vector space S by 
selecting a basis {ai, . . . , cr„}. Clearly 

S = {(j{u) := Midi + . . . + UnCTn : M = (mi , . . . , M„) G M"}. 

With this choice, we fix the Haar measure da on E as the push-forward of the Lebesgue measure 
under the linear isomorphism u i— )■ a{u). By means of this choice we also identify E* with M"^, that 
is, if {al, . . . ,a^} is the dual basis, then we map 

y ^ (^*{y) ■= yi(^*i + ■■■ + ynK- 

The Haar measure da* on S* will be the push-forward of the Lebesgue measure under the linear 
isomorphism y a*{y). Furthermore, for /i G if the action (2.5 ) defines the matrix G GL(n, M) 
in the chosen basis, that is 

h)[a{u)] = a{Mhu) ^ h)[a,] = Y,<^iiMh)ij J = l,...,n, 



and the dual action (2.5) becomes the contragredient of M/^, namely 

h[a\y)] = a*CM-'y). 
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The quadratic map $ given by (2.7) defines n-smooth maps y^i, . . . , : 



by 



In the following, with slight abuse of notation, we write 

(2.9) h^[u] := MhU, h[y] := 'M^'y. 

Finally, we denote by the n x d Jacobian matrix 

d{(pi,...,ipn) 



d{xi, ...,Xd) 



and by J^{x) = a/ det D^{x) *Z^$(a;) the Jacobian determinant. With this notation, the critical 
points of $ are precisely the solutions of the equation J^{x) = 0. The set of critical points is 
invariant under any change of basis in S, whereas the Haar measure da and J$ will change by a 
positive constant, making the condition (2.1) on the admissible vectors invariant. 
In the next two sections we recall the main results of \X\ applied to our setting. 



2.6. Geometric characterization. In what follows, we need a technical assumption that is nec- 
essary in order to avoid pathological phenomena and it is verified in all our examples. To state 
it, we identify ]R"with S* under the map y <H- o'*{y) and hence we regard as an i7-space with 



respect to the second action in (2.9). For any y G M", we denote by H[y] = {h[y] G : /i G H} 
the corresponding orbit and hj Hy = {h E H : h[y] = y} the stability subgroup. The following 
assumption will be made throughout the remaining sections. 

Assumption 1. For every y G M", the orbit H[y] is locally closed in W^. 

The first result gives some necessary conditions in order that G is a reproducing group. These 
conditions become sufficient if the stabilizers are compact. 

Theorem 3. Take G = y\ H E £ . If G is a reproducing group, then 

i) G is non-unimodular; 
a) dimS < d; 

Hi) the set of critical points of ^, which is an H -invariant closed subset ofM.'^, has zero Lebesgue 
measure. 

Furthermore, if dim H = d, then 

iv) for almost every y G $(]R'^) the stability subgroup Hy is compact. 



Conversely, ifvw, nw Hi) and iv) (without assuming dimS = d) hold true, then G is reproducing. 



Proof. Hereafter, all the cited results refer to [T]. Assume that G is reproducing. Theorem 1 
together with Lemma 2 implies [ii]) and iii). Since $ is a quadratic map and the action x h.x is 
linear. Proposition 5 gives Finally, if dimS = d, Theorem 10 yields iv). 

Conversely, Theorem 9, where ^ is understood, proves that G is a reproducing group. □ 
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2.7. Analytic conditions. Because of Theorem [3| in this section we assume the existence of an 
open if-invariant subset X C M'' with negligible complement, whence L'^{X, dx) = L'^{M.'^, dx), such 
that the Jacobian of $ is strictly positive on X. As a consequence, Y := $(A) is an if-invariant 
open set of M", for every y the level set $~^(?/) is a Riemannian submanifold (with Riemannian 
measure dvy) and the Lebesgue measure dy of M", restricted to F, is a pseudo-image measure of 
the Lebesgue measure dx of X (see Lemma 2 in 

The next result is based on classical disintegration formulae of measures. We refer to [5| for the 
general theory. A short account is given in Appendix A of |T]. 



The coarea formula [6] yields the disintegration (2.11) of the Lebesgue measure on X. 

Proposition 4. There exists a unique family {i'y}y^Y of Radon measures on X such that: 

a) for every y eY the measure Vy is concentrated on ^"^{y); 

b) for every ip G Cc{X) 

(2.10) ! v{x)duy{x) = [ ^{x) '^'''^''^ ■ 



X J^'^y) {J^){X) 

c) for every Lp G Cc{X) 

(2.11) / ip{x)dx= ip{x)duy{xj^dy; 



' X Jy ^Jx 

d) for every ip G Cc(A) and h E H 

(2.12) / ip{h-\x)duhiy]{x)=x{h)f5{h) [ ip{x)diyy{x). 

Jx Jx 

The general theory of disintegration of measures gives that 

(2.13) L^{X,dx) = j L^{X,Uy)dy, 

where the direct integral of the family of Hilbert spaces {L^(X, z/j,)}ygy is taken with respect to 
the measurable structure defined by Cc{X) C L^{X,i>y) (see [7] for the theory of direct integrals 



and Proposition 6 in [T] for (2.13)). Accordingly, for any / G L'^{X,dx), we write 



/ = J^fydy, ||/r = JJlfyVdy 

where fy G L^(X, z/^). Since Uy is concentrated on $~^(?/), fy can be regarded as a function on 
$~^(l/). In particular, if / is continuous, then fy is the restriction of / to $~^(?/). 

To state the next disintegration formula, we fix a locally compact second countable topological 
space Z, two Borel maps it : Y ^ Z and a : Z ^ Y, and a Radon measure A on Z such that: 

i) 7r(y) = 7r(t/') if and only if y and y' belong to the same orbit; 

ii) for all z G TriY) we have it{o{z)) = z; 

iii) a set E G Z is A-negligible if and only if tt~^{E) is Lebesgue negligible. 

Remark 2.2. The set Z is a sort of redundant parametrization of the orbit space Y/H by means of a 
locally compact space (in general Y/H is not Hausdorff, hence not locally compact, with respect to 
the quotient topology), vr replaces the canonical projection from Y onto Y/H (though, in general. 



71 is not surjective) and o is a Borel section for vr. Property iii) states that the A-negligible sets 



of Z are uniquely defined by vr and the Lebesgue measure of Y, so that the measure class of A 
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is unique. As a consequence of iii), A is concentrated on 7f{Y). The existence of Z, A, vr and o 
follows from Assumption [l] and Theorem 2.9 in [S], as explained in [I] before Theorem 3. In many 
examples, however, Y/H is indeed locally compact and we may safely assume that Z = Y/H and 
that TT is the canonical projection. As for the measure A, one takes an positive density fonY 
and then builds the image measure of f ■ dy under vr. 

The next result is again obtained from the theory of disintegration of measures (see the proof of 
Theorem 4 of [1] for a list of references). 



Proposition 5. There is a family of Radon measures {tz}z^z on Y with the following properties: 
a) for every z & Z, the measure Tz is concentrated on the orbit tt~^{z) and, for every h & H and 
^ e C,{Y) 



(2.14) / ^ih[y])dTziy) = xih)-' / ^(l/)rfr,(y); 

b) for every ip G Cc(Y) 



Y 



V{y)dy= U (p{y)dTz{y))dX{z) 



As in (2.13), the following direct integral decompositions hold true 

L\Y,dy) = [ L\Y,Tz)d\{z) 



(2.15) L\X,dx) = J{J L\X,Uy)dTz{y)y\{z). 



Y 



liz(^ 7r(r), then = and L'^iY, Tz) = {0}. By ( plsj ), for any / G L^{X, dx) we write 

(2.16) jj' "^^^^^ Jy "'y^'^'^'^y^ 

(2.17) fz = j^f.,ydTziy), fz,y G (X , Uy) . 

In order to write the admissibility conditions, we introduce a unitary operator S that plays a 
crucial role because it implements the diagonalization of U. Some extra ingredient is required. 
Given z G 7r(F), we take o{z) as the origin of the orbit n~^{z) = H[o{z)], which is locally closed by 
assumption. Hence, there exists a Borel section Qz : 7i~^{z) ^ H such that qz{o{z)) is the identity 
element of H and qz{y)[o{z)] = y for all y G H[o{z)]. Since is concentrated on tt~^{z), Qz can be 
extended as a Borel map on Y such that the equality qz{y)[o{z)] = y holds true for Tz-almost all 
y &Y. Furthermore, by (2.8), the level set $~^(o(z)) is invariant under the action of the stability 



subgroup Hz := Ho(z) at o{z), the measure Uz := t'o(z) is concentrated on $~^(o(z)) and, by (2.12) 
it is if^-relatively invariant with weight xP- Finally, we fix the Haar measure ds on Hz in such a 
way that Weil's formula holds true, namely, 

(2.18) J ip{h)x{h)-'dh = j (y ip{qz{y)s)dsyTz{y), veC,{H). 

H Y 
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We begin to build S using the notation introduced in (2.16) and (2.17). Lemma 5 of shows 
that for A-almost every z & Z the operator 



Y 



(2.19) iS^my,x) = VxiqM)(3iqM)fMy)-x), 

is unitary; since A is concentrated on tt{Y), if z ^ ^(^) we set L'^{Y x X, ® i^z) = {0} and 
= 0. It follows that the direct integral operator 

z 



S:L\X,dx)^ / L\Y X X,T,®p,)dX{z) 
Jz 

Sf = J^S^f.dX{z) 



is unitary as well. 



Remark 2.3. As explained in P, 5 diagonalizes the representation U. More precisely, for z G niY) 
define vr^ as the quasi-regular representation of Hz acting on z/^), namely, for s & Hz and 

u e L'^{X, Vz) 



Extend vr^ to a representation of S x if^ by mapping y ^ e 2'r«(?/.o(2)> j^i^ where Id is the iden- 
tity operator on L^(X, z/^). Then, the induced representation Ind^^j:^^(e~^'^*^''°*^^-*^ ® vr^) acts on 
L'^iY X X,Tz® Vz) = Tz'i L'^i.X, Vz)^ and the operator S intertwines U with the direct integral 

representation of G given by Ind^>^j|^^(e~^'^*^''°^^^^ ® tt^) d\{z). 

We are now ready to state the characterization of the admissible vectors. 

Theorem 6 (pQ, Theorem 6). A function i] G L'^{X,dx) is an admissible vector for G if and only 
if for X-almost every z E Z and for every u G L^{X, v^) 

(2.20) \\u\\l= j^i^j^ \j^u{x)S^z{y,s-\x)duz{x)\\{s-'^ms-^)ds) ^^^^^^drziy). 

According to (2.10), the measure Uz lives on $^^(o(z)) and its restriction to this manifold has 
density (J$)~^ with respect to the corresponding Riemannian measure. Moreover, if d = dimE 
the Jacobian criterion implies that $^^(o(z)) is a finite set, Uz is a linear combination of Dirac 
measures and L'^{X, Uz) becomes a finite dimensional vector space. These facts provide a stronger 
version of Theorem [6} which is very useful in the examples. 

Theorem 7 ([T], Corollary 4). Assume that dimS = d. A function rj G L'^{X, dx) is an admissible 
vector for G if and only if for X-almost every z E Z and all points Xi, . . . ,xm ^ 

(2.21) j^r^{h-\xMh-yx,) ^^l^!^^^^^^ = (J<l>)(x,) 5,, z,j = 1, . . . M. 
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2.8. Compact stabilizers. In this section, we make a compactness assumption tliat allows us to 
refine Theorem |6l 

Assumption 2. For almost every y G <I'(]R'^) the stability subgroup Hy of y is compact. 

This assumption allows to decompose U into its irreducible components by means of 5*. Indeed, 
for A-almost every z E Z, is compact, so that the representation tt^ of is completely reducible 
and, hence, we can decompose it as direct sum of its irreducible components 

where each vr^'* is an irreducible representation of acting on some C^^ , any two of them are not 
equivalent, and the cardinal mj G N U {^^o} is the multiplicity of vr^'* into tt^. Theorem 7 in [1] 
shows that it is possible to choose the index set / in such a way that rrii is independent of z (it 
can happen that c/f = 0). Hence, for A-almost all z E Z, 



(2.22) L^Y X X,r, ® z/,) ~ m.L^ (r, r,; C^O • 



Denote by {ej}°^^ the canonical basis of £2 =: C*^" and, for each i G / regard C"' as subspace of 
£2 spanned by the family {cj}™!^. Then, for any / G L'^{X, dx) we write 

mi „ 

(2.23) ^/ = EE / Fi^d\{z)®e, 

iei j=i 

where F*'-' G l^^Y^ t^., C^?) for all z G i G / and j = 1, . . . , m^, and each of the sections z ^ F]^ 
is A-measurable, i.e. for all 9? G Cc(X) the map 

z^ \ Fi^{y,x)S'^{y,x)dn{y)diy,{x) 



YxX 



is A-measurable, where F^^ is regarded as an element of L'^{Y x X, ® t^z) by (2.22) (see remark 
before Theorem 5 and Remark 9 of [T]). Note that, since S is unitary, then clearly 



\f{x)\'dx = Y.Jlj j \\Pny)\\U drMdXiz). 



iei j=i 



Z Y 



We are now ready to characterize the existence of admissible vectors under Assumption |2j 

Theorem 8 (Theorem 9 pQ). ^4 function rj G L^(]R'^, dx) is an admissible vector if and only if for 
all i E I and j = 1, . . . , rrij there exists X-measurable sections 

Z 3 z ^ F'J e L\Y,Tz,C''^) 

such that 

iei i=i i Y 
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and, for almost every z & Z and for all i & I, j,i — 1, . . . ,mi 

(2.24) / £^ = J^^J,. 

Y 

Under the above circumstances 



iei j=i 



3. Orbit equivalence 

In this section we introduce the notion of orbit map and, more importantly, of orbit equivalence, 
which is devised in order to treat in a unified ways families of groups depending on a parameter. 

Suppose wc have two sets of data (S x H, dh, X, Y) and (E' xi H', dh', X' , Y'), each enjoying all 
the properties that we have discussed in the previous section, where dh and dh' are two chosen left 
Haar measures. These data uniquely determine x, /3 and $, and the analogues for the second set, 
which will all be denoted with primed letters. We assume now that the two sets are related by a 
triple of maps {6, called an orbit map between the two sets, which are required to satisfy the 
following properties: 

(OMl) 9 : H ^ H' is an isomorphism between the two connected Lie groups H and H' such that 

dh' = O^dh (the image measure); 
(0M2) -0 : X — )■ X' is a diffeomorphism such that ip{hx) = 9{h)'4>{x) for every h E H and every 

X & X; hence in particular, by invariance of domain, d — d'; 
(0M3) ^ : y ^ y is a diffeomorphism such that ^('l'(x)) = ^'{il){x)) for every x & X; hence in 

particular, by invariance of domain, n — n' . 

Definition 9. An orbit map (6*, V', between (E xi H, dh, X, Y) and (E' xi H', dh', X', Y') is called 
an orbit equivalence if the Jacobian is a positive constant on Y. 

The basic properties of orbit maps {6, -0, recorded below. 

Proposition 10. Let (^,-0,0 orbit map. Then 

(i) ^{h[y]) = 0{h)[$,{y)] for every h & H and every y e Y; 

(ii) = for every y e Y; 

(iii) Ji{h[y])x{h)~^ = x'(6'(/i))~^ J^(|/) for every h E H and every y e Y; 

(iv) Jip{hx)(3{h) = l3'{9{h))Jip{x) for every h E H and every x e X; 

Proof, (i) Using the various properties, including the surjectivity of $ : X ^ F, we infer 

= e($(M) 

= <^'{il){hx)) 
= ^'{9{h)ij{x)) 
= 9{h)[^'{i,{x)] 

-Omiy)]- 
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(ii) This follows immediately from (0M3), since 

$'(^(<l>-i(y)))=e(<f(<f-^(y))) = e(2/). 

(iv) Taking tangent maps at x, we have ip^hxh = 9{h)il)^x- The absolute value of the determinant 
of these yields the equality asserted in (iv). Statement (iii) follows from (i). □ 

Remark 3.1. By (0M3), the diagram 



X 



X' 



Y 



Y' 



is commutative. It is also equivariant with respect to the actions of H (on X and Y) and those of 
H' (on X' and Y'). The equivariance in the left column is (0M2) and that of the right column is 



(i) of Proposition 10 This means that the if-orbits in X are mapped onto the if'-orbits in X' by 
V', and that the if-orbits in Y are mapped onto the if'-orbits in Y' by ^, whence the name orbit 



map. Finally, by (ii) of Proposition 10 fibers relative to $ and $' are mapped onto eachother by 



Remark 3.2. If {6,iIj,^) is an orbit equivalence, then (iii) of Proposition 10 implies that, for all 
h E H, x{h) = x'{0{h)). Thus, two orbitally equivalent data sets must have the "same" x- 

Lemma 11. Suppose that {9,ip,^) is an orbit map. Then, for every y' G Y' , the measure p'y, 
described by Proposition |4] is given by 

(3.1) v'y, = JC^iy) Ji) o ■ ^/>,(z/^-i(^,)), 

where ■?/;=k(i/^-i(j,/)) is the image measure of v^-ii^yi) under ip. 

Proof. We observe en passant that Jif) o is a smooth density and J^~^{y') is of course a real 
number. Theorem 2 in pLj tells us that the family {vy/} is unique provided that it satisfies: 

(i) Vyi is concentrated on $~^(y') for all y' G Y'; 

(ii) the family {vy'} is scalarly integrable with respect to dy' and dx' = jy, Vyi dy'\ 

(iii) for any G Cc(X') the map y' i— )■ ip{x')dh'yi{x') is continuous. 



We therefore show that the measure z/^/ defined by the right-hand side of (3.1) satisfies these 
properties. First we prove that Uy' is concentrated on ($')~^(?/') by showing that such is ip^:{h'^-i(^y>^). 



Indeed, for any Borel set E' C X', if y = ^ ^iv') by (ii) of Proposition 10 we have 

ij-\E')n^-\y) = (ls ^ E'n^{^-\y)) = ^ ^ E' f] {^T\y') = ^■ 

As for (iii), take ip G Cc{X'). Then since x (^(x) = Jip{x) ip{ip{x)) is in Cc{X), by definition of 
image measure we have 



' !f{x')duy>{x') = ^{y') j Ji){x)ip{il){x))dv^-i(^yi 
X' Jx 



\x 



(p{x) duyi^x) 



X 
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The smoothness of ^ and the continuity of the square bracket with respect to imphed by property 
(iii) of the family {uy}, shows the desired continuity. Finally, to prove (ii) we take ip G Cc{X') and 
compute 



Y' JX 



ip{x') duy'^x') dy' 



iy = r\y)) 



(2.11) 



{x' = ip{x)) 



Jy' 



X 



Jip{x)ip{ip{x)) dvi.-^y,){x) 



dy' 



Y 



Jip{x)ip{ip{x)) dvyi^x) 
Ux 

Jip{x)ip{ip{x)) dx 



dy 



X 



f{x') dx' 



X' 



as desired. 



□ 



We introduce the following notation: W : L {X', dx') — )■ L {X, dx) denotes the unitary map 

Wf'ix) = ^/Jij{x)f'{ij{x)). 
We are finally in a position to state our main result on orbit equivalence. 

Theorem 12. Let {6,iIj,^) be an orbit map between (S x H,dh,X,Y) and (S' x H' ,dh' , X' ,Y') 
If T] & L'^{X, dx) is admissible for G = S xi if and if 



(3.2) 



is also in L'^{X, dx), then W ^r]* is admissible for G' = H' xi H' . If {6, ip,C,) is an orbit equivalence, 
then all admissible vectors of G' are of the form W'^rj* for some admissible rj G L'^{X,dx). 

Proof. Take f,T]^ L'^iX, dx) and put, for y and h E H 



^f,viy^^)= / fix)vih ^x)duy{x). 

I X 



By Lemma 1 in [Ijj, is admissible if and only if for every / G L^{X, dx) 




2 dhdy 
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Evidently, an analogous statement holds for the primed set. Take next G L'^{X' ,dx'). By 



Lemma 11 and (iv) of Proposition 10 



X' 



'X' 

{x = r\x')) =JC\y') I f\i^{x))i{e{h)~^{x)) Ji;{x)du^~.^y,^{x) 

Jx 

(0M2) = Jr\y') I Vnx)Vi{h-^x)) 
Jx 



Jipix) 



Now the requirement on Haar measures in (OMl) comes finally into play: 
I . , ,,^|2 dh'dy' f f I I , aih\\\^ 



Y' J H 



'Y' J H 

i^iy) = y) = / \u:vf'yri'{y,h)\ 



x'iemm 



2 1 dhdy 



Y J H 



my) x'iomm 




\ujvf'yr,'{y,h)\ 



Y J H 



where in the last line we have used (iii) of Proposition 10 , namely 

jmx:{e{h)) = jmx'ioih-'))-' = jah^'wxih). 

Observe now that since Uy is concentrated on 



X 



j[x)^^ =dvJx) 



= ^f,v*{y,h). 

We may finally conclude that 

(3.3) ^;|j|^ = / /j.v.^,vv.-fe^ 

Therefore, if {Wrj')* = 77 is admissible, then the right-hand side of (3.3) is equal to ||Vr/'||2 = Wf'Wi- 
Hence, if Wrj' = y/J~^~o~^rj G L'^{X,dx), then rj' G L'^{X',dx') because W is unitary. Under these 
circumstances, the left-hand side of ( 3.3[ ) is thus equal to ||/'||2, and this says precisely that r]' 
is admissible. This proves the first statement of the theorem. The second one is clear because if 
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V'J^ o $ is constant, then Wt]' is in L'^{X,dx) if and only if G L'^{X,dx), so f]' is admissible if 
and only if rj is such. □ 

Remark 3.3. The very end of the proof of the theorem shows that under the assumption that 
{6,ip,^) is an orbit equivalence, that is, that is constant, the two groups G and G' are either 
both reproducing or neither of them is, and in the positive case the admissible vectors are in one- 
to-one correspondence. In the next section we will present many examples in which this situation 
occurs. 



Remark 3.4. If 77 is an admissible vector for G and has compact support, then (3.2) is trivially in 
L^. Hence, if G has an admissible vector with compact support, any group G' belonging to a data 
set for which an orbit map between the data sets of G and G' exists is also reproducing. 

4. Admissible vectors 

For the reader's convenience, we recall the classification of the groups in S2 obtained in [U] using 
a more explicit notation. For t G M we put 



Rt 

and we define the subspaces of Sym(2, 
0" 



cos t sin t 




— sin t cos t 





cosht 
sinht 



sinht 
cosht 



u 
u 

u 

V - 



: u G 



■} 



{ 



-u 



:M,t;GM} ^2 = { 



u 








—u 


u 


V 


V 


u 



u G 



: M, f G 



•■} 



{ 



u 














V 


V 


u 



u G 



: u,v G M}. 



The statement below is a more readable version of Theorem (1.1) in Part I. A comment on the 
notation. In four cases, we index a family of groups by means of a parameter a G [0, -foo]. The 
meaning for a = 00 is that of a limit as a — )■ -|-oo. For example, in the case (2.1) we have 
El X {e^Rct : t G M} = Si X {e'/'^Rs : s G M} so that the limit group is Si x {i?^ : s G M}. 



Theorem 13 (Theorem 1.1 [9J). The following is a complete list, up to Sp{2, 
the groups in S2 with 1 < dimS < 2. 
Two dimensional groups: 

(2.1) Si X {e^Rat : t G M}, a G [0, +00] 

(2.2) S2 X {e^A^t : t G M}, a G [0, +00] 

(2.3) S3X {[10] :tGM} 

(2.4) S3x{e*[l?]:teM} 

(2.5) S3 X ^4,,,] : t G M}, a G [-1,0] 
Three dimensional groups: 

(3.1) Si X {e^R, : t,s G M} 

(3.2) S2 X {e^As : t,s G M} 

(3.3) S3X {[-Jo] :s,teM} 

(3.4) S3 X {[^l 0] : s,t G M} 

(3.5) S3 X {[ : s,t G M}, a G M, a 7^ 1/2 

(3.6) Sf X {e^R^t : t G M}, a G [0, +00] 



i) -conjugation, of 
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(3.1) X {e^A^t : t e M}, a G [0, +oo] 
r5.5;S^x{[i*]:tGM} 
(3.9) Y.iy^{e' WW -.te^] 
Four dimensional groups: 

(4.1) S3 X {[gO] : a,b,ce R, a > 0, c> O} 

(4.2) X {e^R, : t,s G M} 

(4.3) X {e% : t,s G M} 







s 

,(Q + l)t 



]:s,te R}, a G [-1,0] 



Five dimensional groups: 

(5.1) X {[g^] :a,6,cGM, a>0,c>0}. 

We start by ruling out a number of groups in the preceding list, which will turn out to be 
complete. The fact that the other groups in the list are reproducing will be established in the next 
section, where the admissible vectors are computed. 

Proposition 14. The following groups are not reproducing 

(2.1,a = oo) Six{i?t:tGM} 
(2.2,a = cx)) S2x{At:tGM} 
(2.3) S3X {[!?] :tGM} 
(2.5,« = 0) S3X {[1 0] :tGM} 
(3.5,a = l/2), S3X {f-;^* J,] :s,tGM} 
(3.6, a = 00) S^x{i?t:tGM} 
(3.7, a = 00) S^x{At:tGM} 
(3.8) S^x{[i*]:tGM} 
(4.4,« = -l) S^x{[*f]:sGM,t>0} 
(5.1) X {[§ ^] : a,6,c G M, a > 0,0 0} 

Proof. All cases are treated using Theorem |3} We present all the details regarding the first group 
and we simply indicate what happens with the others. 

Consider Si x {Ri : t G M}. Assumption [T] is satisfied because the orbits are trivial. By (2.5), 
for a,t G M, we have Rtyaai] = aai and so by ( |2.4[ ) we get a{Rt) = 1. Hence, since {Rt : t G M} is 
Abelian, by (|2.3[ ), Si x {Rt : t G R} is unimodular. Therefore Si x {Rt : t G R} is not reproducing 
by Theorem |3j 

Assumption [T] is always satisfied for all the remaining groups in the statement, and the case- 
by-case verification is straightforward. The following groups are not reproducing because they are 
unimodular. 



S^ X : t G R} 
S3x{[i?]:tGM} 
S^x{[i*]:tGM} 
The group S3 x { [ J; ] : a, 6, c G 



52 X {At : t G R} 

53 X {[of] : s e R,t> 0} 
a > 0, c > 0} is not reproducing because n 



S^ X {At : t G R} 



d = 2 and for 

every g G M the elements [go] ^ ^s" have stabilizers that are not compact. These points form 
a (redundant) list of representatives for a set of orbits that fills up a set whose complement has 
measure zero. □ 
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We are in a position to state our main result. 
Theorem 15. The following is a complete list of non-conjugat^ reproducing groups in S2: 



{2D.1) 


Si 


X 


{e'Rat 


: t G M}, a G [0, +00) 


{2D.2) 


S2 


X 




■.teR}, ae [0,+oo) 


{2D.3) 


S3 


X 


{[!?]: 


t G M} 


{2D.4:) 


S3 


X 


r re"' 
1 L e( 


:tGM}, «G [-1,0) 


{3D.1) 


Si 


X 


{e*i?, : 


t, s G M} 


{3D.2) 


S2 


X 


{eM, : 


t,s G M} 


{3D.3) 


S3 


X 




: s,te R} 


(3D.4) 


S3 


X 




: s,teR} 


(3^.5) 


S3 


X 




: s,t G M}, a G M \ j 


{3D.6) 


s^ 


X 




: t G M}, a G [0, +00) 


(3D. 7) 


^2 


X 




: t G M}, a G [0, +cx)) 


(3D.8) 


^3 


X 




] : t G M} 


(ADA) 


S3 


X 




a, 6, c G M, a > 0, c> 0} 


{4D.2) 


sf 


X 


{e'Rs : 


t,s G M} 


(4D.3) 


s^ 


X 


{eM, : 


t, s G M} 


(ADA) 




X 


rre- 

I L e 


4i),] :s,tGM},«G(-l 



The proof proceeds by case by case analysis. In the course of the proof we also establish the full 
set of admissible vectors. 

4.1. Two dimensional groups. 

(2.1), a G [0, +00). The elements of Si x Ha, with Ha = {ht := e^Rat : t G M}, are 



{u;t) :-- 



e'Rat 
ue^Rnf e~^Rn 



A left Haar measure of Ha is the pushforward of the Lebesgue measure dt under t ^ ht, and all 
Ha are unimodular. The action (2.5) is ht[y\ = e^^y and hence /3(t) = e^* and x(^) = e~^*. The 
intertwining is $(a;) = — ||xp/2, with Jacobian is J$(x) = Hence we put Xa = R?\ {(0,0)} 
and Ya = (— oo,0). The representation is 

?7(.;i)/(xi,X2) = e-*e™(^?+^')/ (e-*i?_„i(xi, X2)) , 

here and in the following examples the vector (xi,X2) has to be understood as the column vector 
[xl]- These groups are mutually orbitally equivalent. For any fixed a 7^ 0, we define the orbit 
equivalence {9,ip,0 between (Si x Ho,dt,XQ,YQ) and (Si x Ha,dt,Xa,Ya) as follows. First, 
9 : Hq ^ Ha is the group isomorphism 6{e^l2) = e^Rat- Further, we parametrize the elements in Xq 
and Xa with the usual polar coordinates (p, 6) and define ip : Xq ^ Xa by ip{p, 6) = (p, 6' + a log p). 
Finally, ^ is the identity on (— oo,0). Both (OMl) and (0M3) are obvious, and for h = e*/2 G Hq 

^{h.{p,e))=^{e'p,e) = {e'p,e + a{t + \ogp)) = e'Rat{p,e + alogp) = e{h)[ilj{p,e)] 

establishes (0M2). Thus we may restrict ourselves to the case a = and for simplicity we write 
H, X, Y and so on, without the index a = 0. 

1 Within Sp{2,R). 
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The action (2.5 ) is transitive on Y, whose origin is chosen to be o = —1/2 and the corresponding 
stabihzer is Ho = {J2}, hence compact. Therefore the group is reproducing by Theorem [s] We 
shall use Theorem N in order to describe the admissible vectors. The relevant fiber is $^^(0) = S^, 
the unit circle, with measure Uo = d^, upon parametrizing its points by (cos ^, sin 6*). We shall 
therefore identify L'^{X, z/q) with L'^{S^, dO). A smooth section g : y — )■ if for which q{y)[o\ = y is 
q{y) = ^/2\y\I2■ The measure dTo on Y is dy. Because of Weil's integral formula (2.18), the Haar 
measure of Ho must be 61^. Next, 

Sr]{y,9) = 77(^/21^1 cos 6*, ^/2\y\sm9). 

Finally, the quasi regular representation of the singleton group Ho is vr = id on L'^{S^,d6), hence 
it decomposes as the direct sum of countably many copy of the identity on C. Next, we choose 
the exponential basis {{2n)-^/^e'^^ : k e Z} of L'^{S\d9), so that 

F^{y) = ^=f r/(^/2Hcose, v/2^sine)e"^^^d^. 



Theorem [s] tells us that rj G L'^{X, dx) is admissible if and only if 

\F\y)\'dy< +00, [ F\y)V^{y)f^ =25, 
/cgz t'M— \y\ 

(2.2), a e [0, +00). The elements of Si x with if„ = {ht = e^A^t : t G M}, are 

(n;t) := 



e'Ao^t 

-M 1 6 Ant 6 A_ 



A left Haar measure of H^ is the pushforward of the Lebesgue measure dt under t ht and 
Ha is unimodular. The action ( |2.5 ) is ht[y] = e^^y and so (3{t) = e^* and x(^) = e~^*. The 
intertwining is $(a;) = —{xl — x^)/2, with Jacobian J$(x) = ||a;||. Thus, for every a we put 
Xa = \ {(xi, 0:2) : xl — xl = 0} and Ya = M*. The representation is 

f/(«;t)/(xi,X2) = e-*e™(^'?-^'^)/(e-*A_,t(xi,X2)). 

The groups Si x Ha are mutually orbitally equivalent. Fix a 7^ 0. We define the orbit equivalence 
{9,ip,C,) between (Si x HQ,dt,XQ,Yo) and (Si x Ha,dt,Xa,Ya) as follows. First, 6 : Hq ^ H^ is 
the group isomorphism 6'(e*/2) = e^Aat- Now, the four connected components of Xa will be labeled 
by (±, i), with i = 0, 1; the plus sign corresponds to the north-south quadrants and the minus sign 
to the east-west quadrants; labels north and east, and 1 south and west. Each quadrant is then 
fibered by branches of hyperbolae. Thus, a point in a quadrant is parametrized by the branch of 
the hyperbola, labeled by p > 0, and by the real variable s running along it. Explicitly, write 

= p(sinhs, (— l)*coshs) for x in the quadrant (+,i) 
= p((—l)* cosh s, sinh s) for x in the quadrant (— ,^) 

and denote by (p,s) these hyperbolic coordinates on Xq,, see Fig. [l} 

The map ^ is the identity of M*. The map ip : Xq ^ Xa is defined separately on each quadrant. 
For example, in the quadrant (—,0) it is given by ip{p,s) = {p,s + alogp). Both (OMl) and 
(0M3) are obvious, and for h = e^l2 & Hq, 

ilj{h.{p,s)) =ilj{e^p,s) = (e*p,s + a(t + logp)) = e*A„t(p, s + alogp) = e{h)['4j{p, s)] 



(4.1) 
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(+,0) 




Figure 1. Hyperbolic coordinates 



establishes (0M2); the other three quadrants are treated similarly. Thus, we may restrict ourselves 
to the case a = and for simplicty we write H, X, Y and so on. The action (2.5) on Y has the two 
orbits M- = (— oo,0) and IR+ = (0,+oo). As origins we chose ±1/2 with stabilizers H± = {I2}, 
hence compact. Therefore the group is reproducing by Theorem |3j We shall use Theorem [8] in 
order to describe the admissible vectors. The relevant fibers are $~^(l/2), the equilateral hyperbola 
with vertices in (0,±1), and $"^(—1/2), the equilateral hyperbola with vertices in (±1,0). The 



measures h'± on X are concentrated each on the relative hyperbola and, because of (2.10) are given, 
for functions in Cc(M^), by 



(4.2) 



(4.3) 



/,(x„x.)..,(x„x.)^$:/,(smh.,(-l)^cosh.) ds 

/ v'((— 1)* coshs, sinhs) ds, 

i=OA 



with the parametrization (4.1). We shall therefore identify each of L'^{X, z/+) and L'^{X, vJ) with 
two copies of L^(]R, ds). A smooth section q:Y-^Hioi which g(?/)[±l/2] = ?/ is q{y) = \j2\y\l2- 
Weil's integral formula (2.18) forces the Haar measure of H± to be 5. The measure dT± is dy on 
]R±. Next 

S^v±{y.^) = r/±(V2H^), yeY,xe $-^(±1/2). 

Hereafter we shall write any function on X as a sum of four functions, each supported on one of 
the connected components of X. The quasi regular representation vr^ of each singleton group H± 
is the identity on two copies of L^(]R, ds). Choose bases {e'^ of i^^(M, ds), one for each of the 
four basic branches and put F^-{y) 
admissible if and only if 

J2 [ \P±Ay)\'dy<+oo 2 = 0,1 



{S V±iy , ■) , e-!^ i) ■ Theorem 8 tells us that rj E L {X,dx) is 



FUy)F, 



±,k 



(y) 



dy 



je<Jik 



k = 0,1, Z. 
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(2.4). The elements of G = S3 x if, with H = {ht := e^ll^^] : t G M} are 



[u-t) :-- 



ue 






e* 












t 






A left Haar measure of H is the pushforward of the Lebesgue measure dt under t ^ ht and H is 

e^*y, hence /3{t) 



e and x(^) = e . The intertwining 
|a;i|. We put X = \ j^;^ = Qj and Y = (-00, 0). The 



unimodular. The action (2.5) is ht[y] ■ 
is $(a;) = —x1/2, with Jacobian J$(a;) 
representation is 

Uiu;t)f{xi,X2) = e-*e™^V (e-*(xi,X2 - tei)) . 

The action (2.5 ) is transitive on Y, whose origin is chosen to be o = —1/2 with stabilizer Ho 
hence compact. Therefore the group is reproducing by Theorem |3} We shall use Theorem [8] 
in order to describe the admissible vectors. The fiber $~^(— 1/2) consists of two vertical lines 
{(±1,X2) : X2 G M}. The measure z/^ is the push-forward of one-dimensional Lebesgue measure to 
the straight line {(±1,X2)}, and is given by the integral formula 



(4.4) 



X 



(p{l, s)ds + / (p{—l,s)ds, (f G Cc 



whereby we parametrize the lines of the fiber by (±1, s), with s G M. A smooth section q : Y H 
for which q{y)[— 1/2] = y is 

1 0" 
_log(v^) 1. ■ 

The measure dTo on Y is dy. Weil's integral formula (2.18) forces the Haar measure of Ho to be 
the Dirac mass 61^. Next, 



qiy) = 



V 



2|y|(±l,X2±log(v^)) 



Hereafter we shall write any function on X as a sum of two functions, each supported on one 
of the connected components of X, labeled by "+" for the right half plane and by "— " for the 
left half plane. A point in a half plane, in turn, is parametrized by the vertical line to which it 
belongs, determined by y, and by the real variable s running along it, as explained above. Finally, 
the quasi regular representation of the singleton group Ho is it = id on each of the two copies of 
L'^{R,ds). We choose two bases of L^(]R, rfs), one for each of the two basic lines, and put 

F^{y) = {Sri±{y, ■), e-!^)- Theorem [s] tells us that r] G L^{X, dx) is admissible if and only if 



\Fi{y)\'dy < +00 



(2.5), a G [-1,0). The elements of 
are of the form 



S, X Hr. 



\y\ 

with Hn 



e,iG{±}, j,£gZ. 



{h 



t ■- 



diag(e"*, e(°+i)*) 



t G 

























cut 

ue 
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A left Haar measure is the pushforward of the Lebesgue measure dt under t ht and H is Abehan, 
hence unimodular. The action (2.5) is ht\y\ = e^^^y and hence (3{t) = e(^"+-'^)* and x(^) = e~^"*. 
The intertwining is ^(x) = —xf/2, with Jacobian J$(x) = Hence we put = \ {xi = 0} 
and Ya = {—oo,0). The representation is 



U{u;t)f{Xi,X2) 



-at„ 



-{a+l)t 



X2) . 



The groups are mutually orbitally equivalent. For a G (—1,0), we define the orbit equivalence 
{6,ip,^) between (G_i, dt, and ((?„, rft, X^, Fq.) as follows. First, 6 : H^i — )■ is 

the group isomorphism 6'(diag(e^*, 1)) = diag(e"*, e"*^^-'*). Next, if) : X_i — > Xa is given by 
'il}{xi,X2) = {xi, |a;i|^a;2), whereas ^ : F_i — )■ Ya is ^(y) = y. The verification of (OMl), (0M2) 
and (0M3) is straightforward, and evidently = 1. Thus, we may restrict ourselves to the case 
a = — 1 and we write if, X, Y and so on. 

The action (2.5) is transitive on F, whose origin is chosen to be o = —1/2 with stabilizer = {I2}, 
hence compact. Therefore the group is reproducing by Theorem |3} We shall use Theorem [8] 
in order to describe the admissible vectors. The fiber $~^(— 1/2) consists of two vertical lines 
{(±1,^2) : X2 G M}. The measure z/q is again given by the integral formula (4.4), whereby we 
parametrize the lines of the fiber by (±1, s), with s G M. A smooth section q : Y ^ H for which 
q{y)[-l/2]=yis 

'VW\ ol 

1 



Weil's integral formula (2.18) forces the Haar measure of Ho to be Sj^. Next, 



S±v{y,X2) = i2\y\y'/'r] {±y^\,X2 



Hereafter we shall write any function on X as a sum of two functions, each supported on one 
of the connected components of X, labeled by "+" for the right half plane and by "— " for the 
left half plane. A point in a half plane, in turn, is parametrized by the vertical line to which it 
belongs, determined by y, and by the real variable s running along it. Finally, the quasi regular 
representation of the singleton group Ho is n = id on each of the two copies of L^(M, ds). We choose 
two bases {e:?t}jgz of L^(]R, ds), one for each of the two basic lines, and put F^{y) = (^Sr]±{y, ■), e-!^). 
Theorem [s] tells us that r] G L'^{X, dx) is admissible if and only if 



Y.I \Fi{y)?dy<+^ 
dy 



3& 



F^{y)FRy)- 



\y\ 



6ji6,, e,L G {±}, ],£ G Z. 



Three dimensional groups, n = 1. 

(3.1). The elements of Si x H, with H = {ht,g := e^Re : t G M, ^ G [0, 2tx)} are 



' e'Re 
ue^Rfj 





e-'R, 



A left Haar measure of H is the pushforward under the map Mx [0, 2tt) — > H defined by (t, 9) 1— t- ht^ 
of the product Lebesgue measure dtdO, and H is unimodular. The action (2.5) is ht^e[y\ 



e^^y, 
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SO that I3{t,e) = 
Jacobian J^(x) 



(3{t) = e^* and = x(^) = ^ The intertwining is 



72, with 



Hence we put X = \ {x = 0} and Y = (— oo, 0). The representation is 



The action (2.5) is transitive on Y, whose origin is chosen to be o = —1/2, with stabihzer Ho 



{(0, 9), : 6 E [0, 2tt)}, hence compact. Therefore the group is reproducing by Theorem^} We shall 



use Theorem [s] describe the admissible vectors. The fiber is $ ^{o) = {x E 



\x\ 



1} = 



with measure z/q = d9 concentrated on the unit circle. This amounts to the integral formula 

ip{x)di'o{x) = / (y9(cos 6^, sin 6*) (i6', ipeCci^^). 
IX Jo 

We shall therefore identify L^(X, t'o) with L'^{S^,d6). A smooth section q : Y ^ H for which 
q{y)[o] = y is q{y) = a/2|?/|/2. The measure dr is the Lebesgue measure dy. Because of Weil's 
integral formula (2.18), the Haar measure of Ho is d6, whence vol(ifo) = 27r. Next, 

Sri{y, {cos9, sin^^)) = ri{^/2\y\ cos9, >/2\y\sm9). 

The quasi regular representation tt of the group Ho — T on L^(X, dz/o) ~ L'^{S^,d9) is completely 
reducible as the direct sum vr = ©ztt^ on the invariant subspaces sp{efc}, where {ek}kez is the 
normalized exponential basis of L'^{S^,d9), namely ek{9) 



e''yV2Tr. Let F''{y) = (5r/(y, ■), e^). 



Theorem [8j tells us that r] G L {X, dx) is admissible if and only if 

[ J2\^'(y)\'dy<+oo, I \F\y)\ 



2^ 

\y\ 



1 

vr 



We can reformulate these by the natural change of variable r = -\/2|yj. Set ^^^(r) = F^{y), then 



(4.5) 
(4.6) 



/ r dr < +00 



\Vk{r)\ 



,dr 



ike 



d9. 



1 

r 271 

Notice that, under the chosen normalization, 

1 /■^'^ 

?7fc(r) = / r7(r cos 6', r sin 6') e 

v2tt Jo 

the ordinary Fourier coefficient of the restriction of t] to the circle of radius r. 
(3.2). The elements of S2 x H, with H = {hf^g : t, s G M}, are 

e^As 

[ -u ] C*^s 6 

A Haar measure of H is the pushforward of the map — )• if under the map (t, s) 1— )■ /it^s of 



{U]t, s) 



the product Lebesgue measure dtds. The action (2.5) is ht^s[y] = e^*?/, f3{t,s) = f3(t) = e^* and 



x()f:, s) = x('^) = e The intertwining is $(x) = {x^ — x\)/2^ with Jacobian J$(xj 
put X = \ {(xi, X2) G M : Xi = ±X2} and F = M*. The representation is 

f/(n;M)/(^i,a:2) = e-*e™(^?-^^)/ (e-*A_,(xi, X2)) . 



X . We 
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The action ( 2.5[ ) on Y has the two orbits ]R_ = (— oo,0) and ]R_|_ = (0,+oo), whose origins are 
chosen to be ±1/2 with stabihzers H± = {(0,s), : s G M}, hence not compact. Therefore we 
shall use Theorem [6] in order to show that G is reproducing, and then we describe the admissible 
vectors. The relevant fibers are: $~^(l/2), the equilateral hyperbola with vertices in (0,±1), 
and $^^(—1/2), the equilateral hyperbola with vertices in (±1,0). The measures i'± on X are 
concentrated each on the relative hyperbola and are given in integral form by (4.2) and ( 4.3[ ). As 



in Section 



4.1 



we will identify each of L^(X, and L^(X, z/_) with two copies of L^(M, ds). A 
smooth section q± : M-t — )• H for which q±{y)[±l/2] = y is q±{y) = \j2\y\l2- The measure dr is 
the Lebesgue measure dy. Weil's formula (2.18) forces the Haar measure of H± to be ds. Next, 



S^r]±{y,x) =ri±{^/2 



y\x) 



Hereafter we shall write any function on X as a sum of four functions, as in Section 4.1 We will 
also adopt the following notation: 



V+ (?+(?/) (sinh s, (— 1)* cosh s)) = ?7+ ( A/2|y|(sinh s, (— 1)* cosh s 



V- (q'-(y)((— 1)* cosh s, sinh s)) = ?7_ ^A/2|y|((— 1)* coshs, sinhs)^ . 



Similarly, we shall write u±^i{s) to mean the analogous restrictions of m G L'^{X, zz-t) to the various 
hyperbolae. We may now apply Theorem |6| Equation (2.20) reads 



\u\ 



± 



(4.7) 



Jr± Jr ' Jr -^q ^ 
Jr± Jr ' 1 



2 dy 

dp—r-: 



2 dy 

dp — r-r 
2|y| 



where Tp denotes the standard translation on functions. Choose first a function u with support 
in the northern quadrant of X, so that u = m+^q and there is only one summand. By setting 

Gy{p) = r/+,o(y)(-p), we get 



\u\ 



dp 



dy 

2?/ 



I n ii2 dy 

F + ,0 * Gy\\2 



2y 



,5 dy 
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This is readily equivalent to requiring the inner integral to be equal to 1 for almost every ^. A 
similar argument for the other quadrants yields 

(4.8) / ^^^M^rfy = 2, a.e. eeM, 2 = 0,1. 

Jr+ \y\ 



Next we use these in (4.7) and we obtain that the mixed terms must vanish, namely, using Parseval's 
equality in the form {f,Tpg) = J^'^if g){p) 

dy 

dy 



= / l{u±fl,Tpr]±fl{y)){u±^i,T.pr]±^i{y))dp^^^^ 



u^o v±,oiy) ] (p) M ^1 v±,iiy) i-p) dp 



u±}}{0 v±,o{y)iO u^i{-0 v±,i{y){-0 



2|y| 

dy 



2|y| 



t/ ]R t-* IK -I- I y I 



which yields the second set of admissibility conditions 
(4.9) 



v±Ay)i-0 v±fl{ym 



\y\ 



dy = 0, 



a.e. ^ G 



0,1. 



It is not difficult to exhibit vectors r] that do satisfy (4.8) and ( |4.9 ), thereby showing that the 
group is indeed reproducing. 

(3.3). The elements of the group S3 x H, with H = {ha^c '■= diag(a~^/^, c) : a, c> 0} are 


c 




{u]a,c) :-- 











A left Haar measure of H is dh 
by ha,c[y] = CL~^y and hence f3{a,c 



ac) ^dadc and H is unimodular. The action (2.5) is given 

) = a~^/^c and x('2)C) 



$(xi,X2) = —xl/2, with Jacobian J^{xi,X2) = 
The representation is 

U(u;a,c)f{Xi,X2) = a^^^C 



x(«) 



a. 



The intertwining is 



We put X = R'^\{xi = 0} andF = {-00, 0). 



The action (2.5) is transitive on Y, whose origin is chosen to be o = —1/2 with stabilizer given 
{(l,c) : c > 0}, hence not compact. Therefore we shall use Theorem [6] in order to 



by 

show that G is reproducing, and then we describe the admissible vectors. Now, the relevant fiber 
is $^^(—1/2) = {(±1,^2) : X2 G M}. The measure Uo is decribed by (4.4). A smooth section 
q : Y H hr which q{y)[— 1/2] = y is 

'my 0" 
1 



(i{y) 



24 



G. ALBERTI, F. DE MARI, E. DE VITO, AND L. MANTOVANI 



Because of Weil's integral formula (2.18), this forces the Haar measure of Ho to be 2dc/c. Next, 

S^±{y,X2) = {2\y\)-'/^r]{±y^\,X2). 
We may now apply Theorem |6j The right-hand side of (2.20) is 

2dc dy 



/" / E / ^^±ix2my\)-'/'c-'/%±^\,c-'x2)dx. 



c 2\y\ 



± 



dxi dc 

2 

Xf C 



(4.10) 



^{u±,TTcr]{±Xi, ■)) 



dxi dc 
xf C 



where tTc is the non- irreducible representation on L^(]R, dt) of IR+ 

7^J{t)=C-'/'f{c~H). 

In order to handle (4.10), we diagonalize tTc with the unitary Mellin transform. Recall that the 
unitary Mellin transform is the operator M : L^(]R+, dt) — )■ L^(]R, d^) that is obtained by extending 

). Its inverse is defined on L^(]R) fl L^( 
1 



from 



to 



by 



and then extended to the whole of L^(]R). It is immediate to see that 
and hence 

{g,7rJ) = {Mg,M{7rJ)) 
= {Mg,c-'-'-Mf) 

Mg{iy-'^Mf{i)di 

= V^M-^ {MgMf) (c). 

Below we use the following notation: r]i{xi, ■) is the function on IR+ defined by 

t h-> r]i{xi,t) = r]{xi, i-iyt) 

and i]l{xi,^) is its Mellin transform evaluated at ^. Take now u supported in the first quadrant 
{(xi,X2) G X : xi > 0, X2 > 0} and denote by Uj its restriction to the half line {(l,t) : t > 0}, 
a function in L^(]R+,(it). Using the above version of Parseval's identity for the Mellin transform, 
the admissibility conditions given by (4.10) yield 

dxi 



\u\ 



X 



2 • 
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By the fact that A4 is unitary and by Fubini's theorem, this gives 



Similarly one argues for the other three quadrants. Finally, for an arbitrary function, one uses the 
above (and the analogous relative to the left-half plane), and deduces that the mixed terms arising 



from the square modulus appearing in (4.10) must vanish. Altogether, the admissibility conditions 
are therefore 

(4.11) / 4((-l)^x,,0W^W^)^ = 5^M 

Jr+ ^1 

which must hold for almost every G M and all i,j, k,i & {0, 1}. Finally, we show how to meet 



(4.11). Take four orthonormal functions cpe G L (R^, dxi) satisfying 

J xl\(pe{xi)\'^ dxi < +00 



and one positive function a G L^(]R, dC,), and define, for ^ G 



Xia{^)-^/^(!>{xia{0'^) Xi > 

-xia(0"^/V2(-a;ia(^)"^) xi < 

xiai^y'/^ipsixiaiO'') xi > 

-Xia(0~^/^V^4(-Xia(^)~^) xi < 0. 



Finally, define 



r]{xi,X2] 



{r]ly {xi,X2) X2>0 
{r]l)\xi, -X2) X2<0 



where we have denoted by {rjjy^xi, X2) the inverse Mellin transform evaluated at X2 of the function 
defined on M by ^ 1— >■ r^f (xi, ^), for z = 0, 1. We show first that t] G L^{X, dx). Using the fact that 
M. is unitary, this is equivalent to proving that 



/ / x\<^{i) ^Wtixio-ii) ^)\^ dxid^ < +00, 



which follows by the change of variables xi = xia{^) ^ and the hypotheses on (pe and on a. The 
admissibility conditions (4.11) follow from the orthonormality of the (pe. 



4.2. Three dimensional groups, n = 2. The dimension of S in the group (3.4) and in the family 
of groups (3.5) is one, but they are conjugate to semidirect products in £2 for which the normal 
factor has dimension 2. 
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The groups(3.4) and (3.5). Consider the symplectic permutation 

-l' 

-10 ( 

w 





-10 

1 
0-1 









It is easy to show that it conjugates the group (3.4), that is G = S3 x | ['^J °t] : s,t E M}, into 

(4.12) wGw-' = ^iy^{[^;^\] -.teR} 
and the groups (3.5), that is Ga = S3 x | j^*^^* e{<:«+i)t] ■ s,t E M}, into 

(4.13) wG^w-' = S^ X {[^-7''* ,°.] : t e M}. 



Recall that the value a = 1/2 must be neglected because G1/2 is unimodular. Next we relabel the 

cosC = , sin^ 



groups (4.12) and (4.13) in a single family. Upon defining 

a + 1 



a 



we write 



-jt cos C, 



sin 



: teR}, 



Cg[-^,^)\{Co}, 



where tan^o = ~l/3 corresponds to a = 1/2. Notice that if_,r/4 is conjugate to (3.4). We show 
below that all the groups := S3 x H^^ are mutually orbitally equivalent, by exhibiting orbit 
equivalences of each G^ with Gq. 

A left Haar measure on if^ is the pushforward of the Lebesgue measure dt under t 1— /i((t) and 
is unimodular. The actions (2.5) is 



so that (3{t) = e*(™'^f+'^''^^) and x(^cW) = e~*(3smC+cosC)_ ^^^g j^^p <^{x) = {-xl/2, -X1X2) inter- 
twines, and has Jacobian J^{x) = x\. Due to the orbit equivalence that we are about to define, we 
slightly strengthen the natural choice of X and put = M? \ {xiX2 = 0}, and Y"^ = ]R_ x M \ {0}. 
The representation is 



"g2isinf 




yi 


gt(sin f +COS 




y2 



U(u,v;t)f{Xl,X2) 



-t(cos <^+sin Q /2 ni{ux2+2vxiX2)/2 



f{hd-t){x,,X2)). 



The orbits under the action (2.5) all lie in the left half plane and are: open rays issuing from 
the origin for ( ^ {0, — 7r/4}; open vertical rays issuing from the ?/i-axis and open horizontal rays 
issuing from the 1/2-axis in the negative direction if C = ~^/4, see Fig. |2} 
Next, for ( (0, we put 

1 

^ 3sm( + cos ( 
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Figure 2. Left to right: orbits for ( ^ {0, -vr/4}, C = and then ( = -tt/A. 
and define the orbit map from {Gq, dt, Xq, Yq) to (G^, dt, X^, Y^), with C 7^ 0, as the triple 





)= ' 


1 





-jtej cos ( 



^tCQ sin 



I Cf COS r . / \ \ \ct sm C 



1 2/2 



1 2C(; sin 



2/1 sign (ya) 1 2/2 



C(;{sin (^+cos f ) 



Properties (0M1),(0M2) and (0M3) are easily verified. Notice in particular that the mappings 
are smooth in the open quadrants in which they are defined. It is tedious but trivial to check that 
the Jacobian of ^ is equal to 2|c^ sin (^|, hence constant and non vanishing because C 7^ 0. Thus the 
triple is an orbit equivalence and we may restrict ourselves to the case G = Gq and for simplicity 
we write H, X, Y and so on. For any fixed point {yi,y2) € Y, the orbit is 

H[iyi,y2)] = {iyi,e%) : t G M} = {y,} X M±, 

where we take either IR+ or ]R_ according to the sign of y2- As origin of each orbit we chose the 
intersection of the orbit with the half line y2 = ±2/1, that is o{zi, Z2) = (zi,sign(z2)|2:i|), see Fig. Isl 



Figure 3. Orbit representatives 
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We label the elements of Z with s G M*, identifying s with the point o(s) = (— |s|,s) in the 
plane, namely the origin of the orbit. Each stabilizer is the identity of if, hence compact, and the 
orbits are locally closed. Therefore the group is reproducing by Theorem |3j We shall use Theorem 
[8] in order to describe the admissible vectors. 

Fix s ^ Z . An easy computation shows that 



of the orbit if[o(s)] have the form ?/* : = 
for which s)] = yl is 



s , sa 



-i(o(s)) = ±(-sign(s)v^2^, ^2^). The points 
with a > 0. A smooth section : — t- B. 



a 
\s\ 



It follows that 



S'fs{y:,x) = f, 



\S\Xi 



,X2 



The representation vr^ of the singleton group is the identity on 
{61,62} of C^. lir] e L2(M2) and we write := {S'7]s,ej), with j 
and only if 



)|, sign(s)a)F/(— |s|, sign(s)a 
(3.6), a G [0, +00). The elements of Ga 

{u,v;t) : 



-da 
I — 
a 

with H, 



(o(s))) ~ C^. We fix a basis 
1,2, then 77 is admissible if 

?,^= 1,2. 



{ht := e^Rat : t G M}, are 



6 Rat 
V —u 6 -R( 



■at 







at 



The Haar measure on Ha is the pushforward of the Lebesgue measure dt under t ^ ht. 

6^* and x(t) = 6 



contragradient action 2.5 of on M is ht[y] = e'^^R2aty so that /3{t) 



The 

4t 



The interwining is = 
X„ = M2\{(o,o)} and 



{{xl — xl)/2, —X1X2), with Jacobian J<l>(x) 
: \ {(0, 0)}. The representation is 



\x\ 



Hence we set 



U(u,v;t)f{Xl,X2) 



-t^^i{u{xl-xl)+2vx,X2)j^^~t^_^^^^^^^^^Y 



We show that these groups are mutually orbitally equivalent. Indeed, fix a 7^ and define the 
map 6 : Hq ^ by 6{e^l2) = c^Rat- We parametrize the elements in Xq and with the 
polar coordinates {r,6) and define ip : Xq ^ Xa by 'ijj{r,9) = {r,9 + a\og{r / \/2)). Similarly, we 
parametrize the elements in Yq and Y^ with the polar coordinates (p, u) and define ^ : Yq ^ Y^ 



by ^(p,w) = {p,uj + alogp). Observe that in polar coordinates $ becomes 
whereas the actions are 



72,2^ 



(4.14) 



ht[{p, u)] = (6>, u + 2at), ht.{r, 6) = {e^r, 6 + at). 



The verification of (OMl), (0M2) and (0M3) is now straightforward. A tedious but direct com- 
putation shows that = 1. Thus we may restrict ourselves to the case a = and for simplicity 
we write H, X, Y and so on, without the index a = 0. 

l"^ issuing from the origin, 

-7r/2,7r/2] =: Z 



Prom (4.14), with a = 0, it is clear that the orbits in Y are rays in 
hence locally closed. They are parametrized by the angle u = 26 + tt with 6 G 



and A is the Lebesgue measure dO. For each 9, we pick as origin of the corresponding orbit the point 
o{9) = |(cos(2^ + 7r),sin(2^ + 7r)), whose fiber in X is the pair $-i(o(^)) = {±(cos^, sin^)}. The 
group is reproducing by Theorem [3] because it is non unimodular and the stabilizer is a singleton 
for every y ^Y. 
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Now we use Theorem [T] to determine the admissible vectors. A function rj E L^(]R^) is an 
admissible vector for Gq if and only if for almost every 6 G (— 7r/2, tt/2] 

I \ri{±e-\cose,sme))\'^e^^dt = l, ! ?7(e-*(cos^, sin^)-r7(e-*(cos^, sin^) e^* dt = 0. 
A final change of variable r = e~* yields 



/ 

Jr. 



\rj{r cos^,rsin( 



dr 



for almost every 9 G {—11,11], and 



ri{r cos 6, r sin 9)r]{r cos{6 + 7r),rsin(^ + vr)) 



dr = 



for almost every 6 G (— 7r/2, 7r/2]. 

(3.7), a G [0, +00). The elements of Ga '■= xi H^, with if^ 



{/li := e^Aat : t G M}, are 

/I . n " 

(4.15) {u,v;t): = 

The Haar measure on is the pushforward of the Lebesgue measure dt under t y-^ ht. The 
contragradient action 2.5 of Ha on M is ht[y\ = e^*A2aty so that Pit) = e^* and x(t) = The 



interwining is $(x) 



(xf + xl)/2, —X1X2), with Jacobian J$(x) 



^ 1 



Hence we set 



Xa = {ixi,X2) G : X2 7^ ±a;i} and = {(^1,1/2) G : yi < 0, I1/2I < The representation 

is 

Uiu,v;t)f{xi,X2) = e-'e-^'^<-"+^"^'^^^^-'^^^f{e-'A_at{x,,X2)). 

As before these groups are mutually orbitally equivalent. The proof is as in the previous example by 
replacing polar coordinates with hyperbolic ones. Note that X has now four connected components, 
each of them is parametrized by a pair of hyperbolic variables and is mapped one-to-one onto Y . 
We leave the explicit computation to the interested reader. Hence we shall only treat the case 
a = 0. 

The orbits in Y are rays in issuing from the origin with slope between (3/4) tt and (5/4) tt. Each 
orbit is locally closed and we choose as origin the point o(^) = -^(1, sin(2^)) with 6 G (-7r/4, 7r/4). 
Hence Z = {—ir/ 4,11/4) and A is the Lebesgue measure dO. For each 6 the corresponding fiber 
$"^(0(6*)) has four points {Tj (cos 6*, sin 6*) : I <i < 4}, where 



Ti 



To 



1 

1 



-To. 



The group is reproducing by Theorem [3] because it is non unimodular and all stabilizers are 
singletons. 

Now we use Theorem [t] to determine the admissible vectors. A function t] G L^(]R^) is an 
admissible vector for Go if and only if for almost every 6 G (— 7r/4, ir/4) 



1 < i < 4 



[ \r]{e~'Ti{cos e, sin e))\'^ e^' dt = cos(2^), 
Jr 

/ 7]{e'%{cose, sine))r]{e-%{cose, sine))) e^^ dt = I < t < j < 4. 
Jr 
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A final change of variable r = e~* yields 

\ri(Ti{r cos 9, r sin6'))p 



dr = |cos(2^)|, 
TjiTiir cos 6*, r sin 6))rj{Tj{r cos ^, r sin 6')) 



1 < z < 4 



dr = {} 0<i<j<3 



for almost every 6 G (— 7r/4, 7r/4). 

(3.9). The elements of G := x H, with H = {ht := e\l\] : t e M}, are 



(4.16) 



e 


fe* 
f e* (f + 'ut)e* 














The Haar measure on H is the pushforward of the Lebesgue measure dt under t ^ hf. The 



contagradient action 2.5 of if on M is ht[y] = e'^^[2ti]y so that f3(t) = e^* and x(t) = e 



-4t 



The intertwining map is $(x) = (— X2/2, — X1X2), with Jacobian J$(x) = Hence we define 
X = {(xi,X2) G : X2 7^ 0} and y = {(^1,^/2) G : t/i < 0}. The representation is 

Uiu,v;t)f{x) = e-*e-^("^^+2-^^^^)/(e-*(xi -te2),e-*X2). 

The orbits in Y are curves whose parametric equation 7^ : M — t- F is 72 (t) = -e*(t,l)^ where 

2 

z G (0, +00) labels the corresponding origin o{z) = (— y , 0). Clearly, the orbits are locally closed, 
we can set Z = (0, +00) and A is the Lebesgue measure dz. For each z the fiber $^^(0(2;)) has two 
points, namely (0, ±2;). The group is reproducing by Theorem [s] because it is non unimodular and 
all stabilizers are singletons. 

Now we use Theorem [t] to determine the admissible vectors. A function rj G L^(M^) is an 
admissible vector for G if and only if for almost every z G (0, +00) 



\7]{±e^'zi-t,l))\'e''dt = z\ 

A final change of variable r = ze~^ yields 

\r){±{r log^,r))| 



V{e'^z{t, l))r/(-e-*z(t, 1)) e^* dt = 0. 



/ 



dr = 1 



r]{r log ^,r)?7(-rlog ^, 



dr = 



for almost every z G (0, +00). 



4.3. Four dimensional groups, n = 2. We discuss all the details relative to the groups (4.1) 
and (4.4), and then give a quick overview of the other cases. The reason for this choice is that 
example (4.1) exhibits new interesting structural features, whereas example (4.4) is nothing else 
but the shearlet groups and is thus a relevant test of our approach. 



(4.1). The elements of G 
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(4.17) 



[u; a, b, c) : = 



a 











b 


c 








ua 







-ba-^c-^ 














The Haar measure on H is the pushforward of the measure a~^c~^dadbdc under (a, b, c) i-> ha^b,c and 
the modular function is Ania, b, c) = ac~^. The contragradient action of -ff on M is ha^b,c[y] = cl"^ y 
so that f3{a,b,c) = ac, x(t) = The interwining is $(x) = — with Jacobian J$(x) = |xi|. 
Hence we set X = {{xi,X2) : Xi 7^ 0} and Y = (— oo,0). The representation is 

Uiu-aAc)f{x) = (ac)~5e™^i/(a~^a;i,c~^(a;2 - ba'^xi)). 



Clearly, the contragradient action (2.5) is transitive on Y, whose origin is chosen to be o = —1/2 
with stabilizer Ho = {{l,b,c) : 6 G M, c > 0}, clearly isomorphic to the affine group, hence not 
compact. Therefore we shall use Theorem |6] in order to show that G is reproducing, and then 
we describe the admissible vectors. Now, the relevant fiber is ^~^{o) = {(±1,^2) : X2 G M} with 
measure 1^0 = ^+ + ^-, where S± is the push-forward of one- dimensional Lebesgue measure to the 
straight line {(±1,2:2)}. We shall therefore identify L'^{X,Uo) with L'^{R,dx2) © L'^{R,dx2) and 
write f±{x2) := /(±1,X2). 



A smooth section q : Y ^ H is q{y) = h^y^^^ ^. Because of Weil's integral formula (2.18), this 
forces the Haar measure of Ho to be dbdc/c^. Next, 

Sr]±{y,X2) = {2\y\)-'/^r]{±y^\,X2). 



We may now apply Theorem 6 The right-hand side of (2.20) is 



J2 [ u^ix2)i2\y\)-'/'v{±V2\^\ 



X2 =F & 



) dX2 



db dc 



C3 (2|l/|)3/2 



dy 



a^'i=V2|yl 



/ u±{x2)ri{±xi,-^^^) dx". 



dxi db dc 



dxi db dc 



3„3 



3^2 



(4.18) 



2 dxi dbdc 



3„2 



X'{C 



[R+ ' 

where vr^ are the standard reproducing representations of the affine group on L^(M, (ix) 
(4.19) vrf,,)/(x) = c-V2/(^), 
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which are mutually equivalent under the intertwining isometry Tf{x) = /(— x). Separating nega- 
tive and positive frequencies, the corresponding orthogonality relations read 



. I \ / + , f. db dc 

{u,nl,^^f){n+^^f ,u) — 



+ 



di 



Upon selecting m_ = 0, or m+ = 0, respectively, we get the following four admissibility conditions 

|J-2r7(±Xi,±6)P 



(4.20) 



/ 



dxi d^2 



where the signs may vary independently. Using these, the mixed terms in (4.18) must therefore 
vanish, and thus yield the remaining two admissibility conditions 



(4.21) 
(4.22) 



J^2v{xi,^2)J^2v{-Xi, -^2) 
xl 6 



dxi d^2 = 



dxi d^2 = 0. 



Theorem [6] tells us that G is reproducing if and only if there exist vectors r] G L^(]R^) satisfying 
all the above admissibility conditions. It is easy to se that this is the case. First of all, factor 
ri{xi,X2) = '^i(a:;i)'72(a^2)- Then (4.20) becomes 



r]i{±xi)\ 



X, 



dxi 



1^2 (±6) I 

6 



d^2 



and may be met by taking for 772 a usual wavelet, and then by choosing rji to satisfy 

|r7i(±xi)p 



dxi = 1. 



As for (4.21) and (4.22), it is enough to require 

'7i(a;i)%(-a;i) 



X 1 



dxi = 0, 



which can be achieved with support considerations. 

(4.4), a G (0, 1] the shearlet groups. In order to comply with standard notation, we change some 
parametrization. For any 7 < 1/2 we set := S3 x H^, where 



(4.23) 



{ 



v/2 
v/2 u 



} 



a2' 



1 

a" 2 



: a 



> 0, s G M} 



and the action of on the normal abelian factor is 
(4.24) hlAi^^,v)] 
which shows that G^ is indeed the shearlet group. 



a 






u 









V 



ADMISSIBLE VECTORS. 



33 



The correspondence with the notation in Theorem 15 

7 = (l + 2a)/2(a + l) 



IS 



,t/(7-l) 



and the standard shearlet group with parabohc scahng corresponds to 7 = 1/2, i.e., a = 0. As 
explained in Part I, we require that 7 < 1/2 since the groups with 7 G (1/2, +00) are conjugate 
via a Weyl-group matrix to those corresponding to 7 G {—00, 1/2). However, the description of 
the admissible vectors that follows holds for all choices of 7. 

First of all, we show that there exists no orbit equivalence between any pair of different shearlet 
groups. We do this by finding all possible isomorphisms 6 : — )■ and then showing that the 
condition a{h) = a{6{h)), which is necessary in order to have orbit equivalence (see Remark 3.2), 
implies 7 = 7'. We start by observing that each of the groups if^ is isomorphic to the linear group 
consisting of the matrices 



(a, s) = 

They obey the product rule (a, s)(a', s' 
7 ^7' in (-00,1/2]. 



a > 0,s E 



a^-^ s 
1 

= {aa', s + a^~'^s), just as those in H^, see (4.24). Fix now 



Lemma 16. Any isomorphism 9 : — ify has the form 
for some 6 G M and some d ^ Q. 



(a^->' , 



ds + b- 



1-7 



7 



Proof. In the course of the proof, we write S = 1 — 7. By taking derivatives at t = of the 
one-parameter subgroups {(e*,0)} and {(l,t)}, we recognize at once that the Lie algebra Lie(if^) 
has generators 

X 

and their bracket is [X, F] = 6. Similarly, the Lie algebra Lie(if^') has generators given by 
X' = [ q' 0] and Y' = Y with bracket [X',y] = 6'. Let d6 denote the differential of a Lie group 
isomorphism 6 : — )■ Hy, and denote by ^] the matrix representing d6 in the bases {X, y } 
and {X', Y'}. Thus ad — be ^ and since d6 is a Lie algebra homomorphism 



'5 


0" 







1" 








Y = 









S{cX' + dY) = Sd9(Y) = d9{[X, Y]) = [aX' + bY, cX' + dY] 

This forces c = and S = a6'. Therefore 

'6/6' 6l 



{ad - bc)6'Y. 



dO 



d 



d^O,be 



We thus have 

^(exp sY) = e{ 

e{exptx) = e{ 







exp{sd9(Y)) = exp{sdY) 
= exp{tde{X)) = exp(t 



{6/6')6 




sd 
1 

b 
1 



)) 



This means that ^^((l,s)) = {l,ds) and, with a 
follows from 9{{a,s)) = 0((1, s))^((0, a)). 



e*, e{{a,0)) = {b{a^ - l)/6,a^/^'). The result 



□ 
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We now come back to the issue of orbit equivalence. We know that , under the correspondence 



(a, s) -H- /i we have a{h) = a^^^ 



,2-5 



. Since a{h) = a'{6{h)) is assumed to be vahd for every 



h G H^, then ^ = {a^^^')'^ ^' for all a > 0, and this implies 6 = S'. 

Our next goal is to prove that is reproducing. The Haar measure on is the pushforward 
of the measure a'^~^dads under (a, s) i— )■ /ia,s- The contragradient action of on is 



(4.25) 



/ia,s [(1/1,2/2)] 



' a-^ " 




yi 








P. 





a ^yi 



a '^(2/2 - yisaf ^) 



so that /9(a,s) = a"''', x{o.^s) = a^'^^^\ The intertwining map is defined on the whole of by 
^{xi,X2) = —l{xl,xiX2) and has Jacobian J{^){x) = xl/2. We set X = {a; G : 0:2 7^ 0} and 
y = {(2/1,2/2) : 2/1 < 0,2/2 £ IR}- The representation is 



By (4.25) the action (4.25) on Y is transitive and the stability subgroups are singleton. The group 
is reproducing by Theorem [3j 

Our next goal is to compute the admissible vectors. In order to illustrate the two possible 
approaches arising (in this case) from the general theory, we shall perform the computation both 
ways: via Theorem [7] and via Theorem |8] 

First method. Clearly Z = {0} and A is Sq. We select the origin o = (—1/2,0). Evidently, 
$-i(o) = {±(0, 1)}. Condition ([2?2l]) becomes 



\ri{± h^l (0, 1)) |2 a^-'^ dads = / \ri{± {sa^-^^^, a^^^))\^ a?-^ dads. 



For simplicity, we introduce the maps F 



± . 



defined by T^{a, s) = ± {sa'^ a^^"^), whose 



Jacobians are J{V^){a,s) = a^ ^/2. Here = {{xi,X2) G : ±0:2 > 0}. The above relation 
can thus be rewritten as 



1 
2 



|r/(F±(a,.))p 



2J(F^)(a, s) 
(ai/2)4 



dads 



Similarly, condition (2.21) for the pair {Pi, P2) yields 

= 



r]{xi,X2)ri{-Xi, -X2) 



1^(3^1,3:2)^ 
4 
X2 



dx. 



dxidx2- 



In conclusion, a function T] G L2(]R2) is an admissible vector for if and only if 

\r]{xi,X2)\^ 



(4.26) 



dxidx2 



1 



vixi,X2)r]{-Xi,-X2) 



X2 



dxidx2 = 0. 



Second method. Next, we re-obtain the above conditions by applying Theorem [8} that is, by 
decomposing the restriction of the metaplectic representation to G^ into its irreducible components. 
The first step of this process consists in applying Proposition [4], thereby obtaining the disintegration 
formula dx = jyVydy for the measure dx on X. For ?/ G F we have $^^(2/) = {Pf,P2} with 
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Pf = V^{y2/ ^/\yl\^ \Ay^\) ^^'^ ^2 ~ To compute the measures Vy we use (A. 9) in [T] 

recalling that Vy is concentrated on {Pf , -Pj'}, and we obtain 

p ( py\ = I = — 



In particular, for each of the inverse images P° of the origin o we have z/o(Pj) = 2. As in Section 2.7 



we write / = fyfydy for every / G L^{X,dx) = jy L'^{X,Uy) dy. Since Z = {0} the second 
disintegration discussed in Proposition [5] is trivial and we simply get dr = dy. 

The second step is to write the operator S that intertwines the metaplectic representation with 
the relevant induced representation. An easy computation shows that the map 

g: y — )■ H-y, q{y) = h^_ y2_ 

2|yi I ' lai 

satisfies q{y)[o\ = y and so q{o) = I. The map q gives a diffeomorphism between Y and and 
has Jacobian Jq{y) = l/2||/i|^. By ( |2.18[ ) we obtain for Lp G Cc{H) 



ip{q{y))vo\{Ho)dy = / (p{ha,s)a{ha,s) ^ dads 

^{ha,s)0'~^ dads 

v{(i{y))8\yifJq{y)dy 
^{(i{y)) dy, 



Y 



Y 



whence yo\{Ho) = 4. Since z/g is concentrated on {P^jPg}; orthonormal basis for L^(X, z/q) 
is {xi/"\/2, X2/v^} where Xi is the indicator function of P°. Hence the components of the map 
S: L\X, dx) L\Y, dy; L\X, i^^)) defined in (|2l9| are 



(SvUy) = ^a{q{ymq{y))Vy{q{y)-P-) = {2\y,\r'/\{P!). 



where rj = fyVydy ^ L'^{X,dx). We now use the decomposition discussed in Section 2.8 Since 
the stabilizer Ho = {/} is trivial, the representation vr of Ho on L'^{X, uq) decomposes as 

vr = 2idc = idc©idc, ^^(X, z/q) = 2 C = C © C. 

Looking at (2.23), we have P* = \/2{Sr])i because {xi/v^, X2/v^} is an orthonormal basis. 
Finally, we are in a position to apply Theorem [s] Take rj G L'^{X,dx) and write P* as above. 
Relation (2.24) becomes 

Y 

= 2 [ {{SvUy),{Sv)j{y))\2yir'dy 



Y 



2 / {vyiPnMPj))\'^yirdy. 



Y 



36 G. ALBERTI, F. DE MARI, E. DE VITO, AND L. MANTOVANI 

If i = j, with the change of variable y = ^{x) we get 



1 
4 



2 / \ri{xi,X2)\ X2 J^{xi,X2)dxidx2 



\V{X1,X2)\' 



dxidx2- 



The orthogonahty condition in (4.26) is obtained similarly. 

e*Re ■.teR,9e [0,2n)} are 

e^Rf) 



(4.2). The elements of G = x {ht,e := 

{u,v;t) :-- 



[v -u]e*-Re e ^Rg 



2t 



The Haar measure on H is the pushforward of the measure dtd9/2'K under {t,9) ^ htfi and H 
is unimodular. The contragradient action of H on is htfi[y\ = e^^R2ey so that f3(t,9) = e 
and = e~^*. The interwining is $(a;) = ((x^ — xl)/2, —X1X2) with Jacobian J<l>(a;) = ||a;||' 

Hence we set X = 



\ {(0,0)} and 



\ {(0,0)}. The representation is 



t/(n,.;i,e)/(xi,X2) = e-*e'^^("(^i-^2)+2^^^^^)/(e"*/?_e(xi,X2)). 

The action (2.5) is transitive on Y. We choose as origin o = —(1/2,0), whose compact stabilizer 
is Ho = {I, Rt^} and whose fiber in X is the pair $~^(o) = {±(1, 0)}. The group is reproducing by 
Theorem [3] and we use Theorem [t] to determine the admissible vectors. Since —(1,0) = i?7r(l,0), 
a function T] e L2(]R2) is an admissible vector for G if and only if 

/ |r7(e-*(cos^,-sin^))|V*^ = l 

Jrx[o,+oo) 27r 

tdtd9 



r]{e-\cos9, - sin^))f/(-e'*i?_0(l, 0))e 



2r 







Rx[0,+oo) 



The change of variables (xi, X2) = e *(cos 9, — sin 9), whose Jacobian is e , yields 



|r/(x)p 



dx 



\x\ 



271 



rj{x)rj{—x) 



dx 



\x\ 



0. 



(4.3). The elements of G = x {ht^^ ■= : t, s G M} are 

e'A,t 



iu,v;t) :-- 

The Haar measure on H is the pushforward of the measure dtds under [t, s) t— )■ ht^s and H is 



-it 



unimodular. The action (2.5) of H on M is ht,s[y] = e y so that j3{t, s) = e and xi^, s) = e 
The interwining is ^{x) = {—{xf + X2)/2, —X1X2), with Jacobian J$(x) = \x1 — xl\. 

We set X = {{xi,X2) G : 7^ ±Xi} and y = {(1/1,1/2) G : 1/1 < 0, I2/2I < |l/i|}, and the 
representation is 



U{u,v;t,s)f{Xl,X2) 



Clearly, the action (|2.5|) is transitive and free on Y, whose origin is o = —(1/2,0) and the corre- 

M, = -n 



sponding fiber in X is $ ^(o) = {Mj(l, 0) : i = 0, . . . , 3} where 



Mn 



J, Ml = n, M2 
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must be seen as isomorphisms of M^. 

The group is reproducing by Theorem |3] and we use Theorem [7] to determine the admissible 
vectors. Set C = {{xi,X2) : Xi > 0,\x2\ < Xi} and observe that the maps (t,s) i— )■ (xi,X2) = 
e~^AsMi{l,0) are diffeomorphisms from onto MjC with Jacobian e~^*. Hence, reasoning as in 
the previous example, a function T] G L2(]R2) is an admissible vector for G if and only if 



I \v{x)\' 




i = 0, ...,3 



/ 7]{Mix)r]{M,x) ^ = 0<t<j<3, 

Jc Fl ^21 

where in the second set of equations we use that Mj are unimodular maps of C onto MjC. 
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